Skip to content

Conversation

@gavinxgu
Copy link

@gavinxgu gavinxgu commented Jun 13, 2025

tiktok/pnpm-sync#40
tiktok/pnpm-sync#43

Summary

Upgrade pnpm-sync-lib to v0.3.3 for pnpm v10 compatibility

Details

1. Upgrade pnpm-sync-lib to Version 0.3.3

Updated package.json to upgrade pnpm-sync-lib from the previous version to 0.3.3, ensuring full compatibility with pnpm v10. This change builds upon the foundation established in tiktok/pnpm-sync#40 (pnpm v9 support) and tiktok/pnpm-sync#43 (pnpm v10 support).

2. Adopt Official pnpm readWantedLockfile Methods

Integrated official pnpm methods in BaseInstallManager.ts to support both v6 and v9 lockfile formats:

// Via rush-pnpm-kit-v8 and rush-pnpm-kit-v9 packages
const lockfileV9: ILockfile | null = await pnpmKitV9.lockfileFs.readWantedLockfile(
  pnpmLockFolder,
  options
);

const lockfileV6: ILockfile | null = await pnpmKitV8.lockfileFs.readWantedLockfile(
  pnpmLockFolder,
  options
);

This implementation provides backward compatibility while supporting the latest pnpm lockfile formats, with automatic detection based on lockfile version.

3. Enhanced Multi-Version pnpm Support

During the upgrade to pnpm v10, packages like @pnpm/lockfile-file and @pnpm/lockfile.fs have peer dependency requirements for @pnpm/logger, but these dependencies were not being properly satisfied, causing warnings or errors during installation.

Solution Implementation

The solution leverages the pnpm-kit architecture - a set of version-specific wrapper packages that encapsulate pnpm functionality for different pnpm versions. As part of this upgrade, three new rush-pnpm-kit packages were added (@rushstack/rush-pnpm-kit-v8, @rushstack/rush-pnpm-kit-v9, and @rushstack/rush-pnpm-kit-v10) to provide comprehensive support across pnpm versions. This modular approach allows Rush to support multiple pnpm versions simultaneously while ensuring proper dependency resolution.

1. Adding @pnpm/logger as Direct Dependencies

@pnpm/logger has been added as a direct dependency in each pnpm-kit package:

libraries/rush-pnpm-kit-v8/package.json

{
  "dependencies": {
    "@pnpm/dependency-path-pnpm-v8": "npm:@pnpm/dependency-path@~2.1.8",
    "@pnpm/lockfile-file-pnpm-lock-v6": "npm:@pnpm/lockfile-file@~8.1.8",
    "@pnpm/logger": "~5.0.0"
  }
}

libraries/rush-pnpm-kit-v9/package.json

{
  "dependencies": {
    "@pnpm/dependency-path-pnpm-v9": "npm:@pnpm/dependency-path@~5.1.7",
    "@pnpm/lockfile.fs-pnpm-lock-v9": "npm:@pnpm/lockfile.fs@~1001.1.11",
    "@pnpm/logger": "~1001.0.0"
  }
}

libraries/rush-pnpm-kit-v10/package.json

{
  "dependencies": {
    "@pnpm/dependency-path-pnpm-v10": "npm:@pnpm/dependency-path@~1000.0.9",
    "@pnpm/lockfile.fs-pnpm-lock-v9": "npm:@pnpm/lockfile.fs@~1001.1.11",
    "@pnpm/logger": "~1001.0.0"
  }
}
2. Modular Encapsulation and Export

Each package exports similar modules with version-appropriate implementations:

export * as lockfileFs from './lockfileFs';
export * as logger from './logger';
3. Usage in Other Packages

In libraries/rush-lib/src/logic/base/BaseInstallManager.ts, different kit versions are imported to handle different lockfile formats:

import * as pnpmKitV8 from '@rushstack/rush-pnpm-kit-v8';
import * as pnpmKitV9 from '@rushstack/rush-pnpm-kit-v9';

// Used in pnpmSyncPrepareAsync call
readPnpmLockfile: async (lockfilePath: string, options) => {
  const pnpmLockFolder: string = path.dirname(lockfilePath);

  // First attempt to read v9 format lockfile
  const lockfileV9: ILockfile | null = await pnpmKitV9.lockfileFs.readWantedLockfile(
    pnpmLockFolder,
    options
  );

  if (lockfileV9?.lockfileVersion.toString().startsWith('9')) {
    return lockfileV9;
  }

  // If not v9 format, attempt to read v6 format
  const lockfileV6: ILockfile | null = await pnpmKitV8.lockfileFs.readWantedLockfile(
    pnpmLockFolder,
    options
  );

  if (lockfileV6?.lockfileVersion.toString().startsWith('6')) {
    return lockfileV6;
  }

  return undefined;
}

How it was tested

Impacted documentation

@gavinxgu
Copy link
Author

@iclanton Could you please review my code? Thank you!

@iclanton iclanton moved this from Needs triage to In Progress in Bug Triage Jun 18, 2025
@gavinxgu gavinxgu force-pushed the guxiang/upgrade-pnpm-sync-lib-to-0.3.3 branch 3 times, most recently from e13d10a to 8790575 Compare June 20, 2025 01:30
@gavinxgu gavinxgu force-pushed the guxiang/upgrade-pnpm-sync-lib-to-0.3.3 branch from 8790575 to 8055d31 Compare June 20, 2025 04:11
@gavinxgu gavinxgu requested a review from octogonz June 21, 2025 06:43
@aramissennyeydd
Copy link
Contributor

@gavinxgu Any updates updates here? Would love to see pnpm 10 support land ❤️

…de-pnpm-sync-lib-to-0.3.3

# Conflicts:
#	apps/lockfile-explorer/package.json
#	apps/lockfile-explorer/src/utils/shrinkwrap.ts
#	common/config/rush/browser-approved-packages.json
#	common/config/subspaces/build-tests-subspace/pnpm-lock.yaml
#	common/config/subspaces/build-tests-subspace/repo-state.json
#	common/config/subspaces/default/common-versions.json
#	common/config/subspaces/default/pnpm-lock.yaml
#	common/config/subspaces/default/repo-state.json
#	libraries/rush-lib/package.json
#	libraries/rush-lib/src/logic/base/BaseInstallManager.ts
#	libraries/rush-lib/src/logic/pnpm/IPnpmfile.ts
#	libraries/rush-lib/src/logic/pnpm/PnpmShrinkWrapFileConverters.ts
#	libraries/rush-lib/src/logic/pnpm/PnpmShrinkwrapFile.ts
#	libraries/rush-sdk/package.json
Comment on lines +1543 to +1559
{
"packageName": "@rushstack/rush-pnpm-kit-v8",
"projectFolder": "libraries/rush-pnpm-kit-v8",
"reviewCategory": "libraries",
"shouldPublish": true
},
{
"packageName": "@rushstack/rush-pnpm-kit-v9",
"projectFolder": "libraries/rush-pnpm-kit-v9",
"reviewCategory": "libraries",
"shouldPublish": true
},
{
"packageName": "@rushstack/rush-pnpm-kit-v10",
"projectFolder": "libraries/rush-pnpm-kit-v10",
"reviewCategory": "libraries",
"shouldPublish": true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iclanton @dmichon-msft I've fixed up this branch to incorporate the latest work from main. Let's try to get it merged.

Besides testing for regressions, the main thing we should consider is the 3 new NPM packages being created.

options
// TODO: pnpm-sync-lib.d.ts was at some point generalized to support multiple lockfile formats,
// however its API still returns a single "ILockfile" that is incompatible with the newer interfaces
)) as ILockfile | null;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point we need to update pnpm-sync-lib to handle this better. A simple solution is to treat ILockfile as unknown or just having the lockfileVersion member. (A complex solution is to depend on @rushstack/rush-pnpm-kit-v#.)

{
"tagName": "@jsxImportSource",
"syntaxKind": "block"
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iclanton was this overlooked from PR #5440?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

4 participants